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This study used the Delphi method to engage 
expert practitioners on the topic of threshold 
concepts—core ideas and processes in a 
discipline that students need to grasp in order 
to progress in their learning, but that are often 
unspoken or unrecognized by expert 
practitioners—for information literacy. A 
panel of experts considered two questions: 
First, is the threshold concept approach useful 
for information literacy instruction? The panel 
unanimously agreed that the threshold concept 
approach holds potential for information 
literacy instruction. Second, what are the 
threshold concepts for infonnation literacy 
instruction? The panel proposed and discussed 
over 50 potential threshold concepts, finally 
settling on six information literacy threshold 
concepts. 
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Introduction 

The oracle at Delphi was a bit like a 
reference librarian, albeit with goat 
sacrifices, trance-induced possession, and a 
much fancier desk. She fielded tough 
questions from kings and commoners alike, 
all of whom sought her ability to channel 
Apollo and reveal the future. Like any 
psychic—or librarian—worth her salt, the 
oracle’s advice was open to interpretation, 
but she always did her best to answer the 
question. This study seeks the wisdom of 
our own oracles, those prominent voices in 
our field, to help us determine the potential 
of threshold concepts for information 
literacy. 

Threshold concepts are one way to approach 
the core concepts in our discipline. They are 
an exciting approach to re-engaging with 
teaching content because they offer a unique 
perspective by which to prioritize 
disciplinary knowledge. While the idea of 
threshold concepts has entered the national 
discourse about information literacy 
instruction via ACRL’s new Framework for 
Infonnation Literacy for Higher Education 
(ACRL, 2014), it remains an emerging 
theory, both broadly speaking and with 
respect to our discipline. As practice- 
oriented professionals, librarians are very 
interested in producing reusable materials 
that incorporate threshold concepts into 
instruction sessions, syllabi, and course 
materials. Using the Delphi method, a 
qualitative approach in which a small group 
of experts anonymously answer questions in 
writing, this study’s goal is to come to 
confident conclusions about the theoretical 
underpinnings of the materials we would 
eventually like to produce and share. 
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This study invited expert practitioners to 
answer two questions. First, are threshold 
concepts useful for information literacy? 
The short answer to this question was yes; 
the in-depth results from the discussion that 
emerged over this question will be 
addressed in a separate publication. This 
paper addresses a second question: What are 
the threshold concepts for information 
literacy? 

Literature Review 

Threshold Concepts 
Threshold concepts —an approach to 
teaching and learning developed by Jan 
Meyer and Ray Land, British educators 
working on a project called Enhancing 
Teaching-Learning Environments in 
Undergraduate Courses—are core ideas and 
processes in a discipline that students need 
to grasp in order to progress in their 
learning, but that are often unspoken or 
unrecognized by expert practitioners. As 
described by Meyer and Land, threshold 
concepts have five definitional criteria: 

• Transfonnative: cause the learner 
to experience a shift in perspective; 

• Integrative: bring together separate 

concepts (often identified as 
learning objectives or 

competencies) into a unified 
whole; 

• Irreversible: once grasped, cannot 
be un-grasped; 

• Bounded: may help define the 
boundaries of a particular 
discipline, are perhaps unique to 
the discipline; 

• Troublesome: usually difficult or 
counterintuitive ideas that can 
cause students to hit a roadblock in 
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their learning (Meyer & Land, 

2003). 

Since this model was developed, Meyer and 
Land have published several books 
exploring threshold concept theory and its 
applications in a wide variety of disciplines 
and learning settings (Meyer & Land, 2006; 
Land, Meyer, & Smith, 2008; Meyer, Land, 
& Baillie, 2010). A biennial conference in 
the UK has served to further develop a 
community of practice and spread new 
thinking in this area. Mick Flanagan 
maintains a bibliography on threshold 
concept publications (Flanagan, 2014). 

At the same time, threshold concepts may 
be understood as a repackaging of many 
other current educational theories, and have 
been shown to work well in tandem with 
them. For example, Lundstrom, Fagerheim, 
& Benson (2014) used threshold concepts in 
combination with Decoding the Disciplines 
(Middendorf & Pace, 2004) and backward 
design (Wiggins & McTighe, 2005) in order 
to revise learning outcomes for information 
literacy in composition courses at Utah State 
University. This flexible approach to theory 
captures a point that researcher Glynis 
Cousin makes: “There are clear overlaps 
and affinities with a number of the ideas 
shared by the theory of threshold concepts 
and other perspectives in education” (2008, 
p. 261). Threshold concepts may be 
understood as a shortcut through the 
theories for disciplinary faculty who do not 
hold advanced degrees in education (Meyer 
& Land, 2007). 

Not surprisingly, with greater dissemination 
and the increase of discourse on the topic, 
positions against threshold concepts have 
emerged. Some critics point out that 
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threshold concepts as a pedagogical theory 
are not proven to be effective per positivist 
methodologies, or that the criteria for 
establishing which concepts are threshold 
concepts are inexact (e.g., Rowbottom, 
2007; Wilkinson, 2014). Librarians’ 
objections often center on ACRL’s use of 
emerging theory to underpin their new 
standards document (e.g., Saracevic, 2014). 

Identifying Threshold Concepts 
Barradell (2013), in her review of methods 
used to identify threshold concepts, finds 
that the threshold concept literature turns up 
a wide variety of methods used to identify 
threshold concepts in different disciplines: 
“informal, semi-structured, phenomenographic 
interviews..., questionnaires, surveys, short 
answer problems and review of old 
examination papers..., and observation of 
classroom behavior” (Barradell, 2013, p. 
269). Barradell also asserts that “The 
conversations in which threshold concepts 
are discussed are recognized as being 
integral to the process” (2013, p. 269). 
Barradell concludes that consensus 
methodologies such as Nominal Group 
Technique and the Delphi method can be 
effectively deployed in order to obtain 
collaborative and structured conclusions to 
these discussions. 

Shinners-Kennedy and Fincher (2013) spent 
years pursuing a multimodal research 
project to identify threshold concepts for 
computer science that included students as 
research subjects, in semistructured 
interviews, concept mapping, and 
journaling. Yet they conclude that their very 
thorough efforts resulted in an unexpected 
dead end, which led to a re-evaluation of 
their methods; in their analysis they found 
that both hindsight bias and a false 
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hypothesis about the emotional state of a 
student acquiring a threshold concept made 
their results too limited. Where Barradell 
(2013) advocates strongly for including 
research with students in developing 
threshold concepts, Shinners-Kennedy and 
Fincher found that it was more productive to 
ask “Where would we look to see evidence 
of threshold concepts in teachers’ practice, 
in their pedagogical presentation of 
concepts, rather than in learners’ acquisition 
of them?” (p. 13). They use a content 
representation fonn developed by Loughran, 
Berry, and Mulhall (2006) as a concrete 
method of capturing instructors’ expertise as 
situated in classroom experience. Their 
work supports the idea that instructors are 
the experts on the threshold concepts for 
their fields. 

The Delphi method is a good fit to validate 
the threshold concept approach for 
information literacy instruction and define 
the threshold concepts for information 
literacy because threshold concepts are 
identified by subject experts. Delphi studies 
have been used in other fields to identify 
threshold concepts. Examples include 
occupational therapy (Nicola-Richmond, 
2014); sustainable agriculture (Nguyen, 
2012); and community service (Fuzzard & 
Kiley, 2013). The authors note, too, that 
Delphi studies have often been used by 
librarians and infonnation professionals 
(examples may be found in Buckley, 1994; 
Baruchson-Arbib & Bronstein, 2002; and 
Feret & Marcinek, 2005). There are also 
many instances in the literature of using the 
Delphi method to research infonnation 
literacy topics—in fact, a Delphi study 
contributed to the definition of the term 
"information literacy" (Doyle, 1992; Green, 
2000; Neuman, 1999; Saunders, 2009; 
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Howze & Dalrymple, 2004; Dixon-Thomas, 
2012; Seeker, 2011). In a number of 
countries, Delphi studies have been used to 
guide development of infonnation literacy 
standards documents at the national level 
(for example, Seeker & Coonan, 2012; 
Xiaomu, Ping, Mengli, & Weichun, 2008; 
Wen & Shih, 2006). 

The literature strongly suggests that 
researchers and instructors may arrive at 
differing or complementary conclusions 
regarding the threshold concepts for a field. 
For example, Buehler and Zald (2013) look 
at learning thresholds that must be crossed 
by graduate students entering the scholarly 
conversation as authors or presenters and 
write that “The publication process can be 
identified as an information literacy 
‘threshold concept’ with particular 
immediacy for graduate students” (p. 219). 
Kiley and Wisker’s related work looks at 
interdisciplinary threshold concepts for 
graduate students learning to become 
researchers in their doctoral programs 
(2009). The equivalent threshold concept 
identified in their study is “knowledge 
creation”: “Supervisors can report evidence 
of the contribution of new ideas and 
thought, and in self-motivated research that 
indicates not just a willingness to work but 
an engagement with the essential issues and 
the leading edge work on the field” (p. 438). 
These findings support the idea that there 
are many learning thresholds associated 
with infonnation studies and infonnation 
literacy. 

Threshold Concepts for Information 
Literacy 

Townsend, Brunetti, and Hofer (2011) 
suggest that threshold concepts can be used 
to prioritize teaching content for 


[Article] 


26 





Townsend, et al, Identifying Threshold Concepts 

information literacy and to develop a 
reflective teaching practice. In a special 
issue of Communications in Information 
Literacy dedicated to the ACRL Information 
Literacy Standards revision, Hofer, Brunetti, 
and Townsend recommend that the new 
standards use learning theories such as 
threshold concepts to focus on disciplinary 
content in information literacy rather than 
procedural how-tos (2013). The revised 
ACRL Framework for Information Literacy 
for Higher Education (the Framework) 
looked to threshold concepts as a way to 
shift the profession’s attention away from a 
checklist approach and toward underlying 
concepts that students need to understand in 
order to become information literate. 
Though the final draft of the Framework 
moved away from using this tenninology, 
threshold concepts continue to infonn the 
document (ACRL, 2014; author Lori 
Townsend was a member of the Task 
Force). 

The profession as a whole may now be on 
the steep side of the learning curve when it 
comes to understanding threshold concepts; 
as Oakleaf (2014) points out, “For many 
librarians, threshold concepts are unfamiliar 
constructs, represent a different way of 
thinking about instruction and assessment, 
and require a concerted effort to integrate 
into practice.” It is not surprising that 
librarians might initially struggle to 
integrate and apply this new approach: “The 
idea of a threshold concept is in itself a 
threshold concept” (Atherton, Hadfield, & 
Meyers, 2008, p. 4). The professional 
community has responded with a wealth of 
conference sessions, workshops, webinars, 
and other learning opportunities for 
librarians seeking development in the area 
of conceptual teaching and learning. 
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Well before the publication of the new 
Framework drafts, instruction librarians 
explored the area of threshold concepts for 
information literacy. Several works 
understand infonnation literacy itself to be a 
learning threshold, as captured by Bent, 
Gannon-Leary, and Webb (2007): “We can 
see that to develop as an information literate 
person, an individual must cross a threshold 
in their attitude to and understanding of 
information in their personal research 
environment” (p. 84). This understanding is 
echoed in Yorke-Barber et al. (2008) and 
Rodrigues and Sedo (2008). 

The authors, by contrast, understand 
information literacy to be a field for which 
there are multiple learning thresholds 
(Hofer, Townsend, & Brunetti, 2012); this 
view of infonnation literacy is shared by 
others with an interest in threshold concept 
research. Margaret Blackmore, for instance, 
developed learning thresholds for 
information literacy by enlisting support 
staff at her institution to identify content 
that is troublesome for students (2010). In 
subsequent work, Blackmore and Freeland 
(2014) argue that infonnation literacy 
should not be taught as a linear series of 
competencies, often limited to search 
strategy. They developed an assignment for 
undergraduates in a game design course that 
begins to put this approach into practice 
through authentic assessment. 

Virginia Tucker’s doctoral thesis work 
(2012) uses a threshold concept approach to 
study the differences between expert and 
novice searchers in order to better 
understand the acquisition of expertise. 
Tucker’s work is able to look at liminal 
spaces because her subjects were “intent on 
becoming experts” (p. 3). Threshold concept 
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theory is well-suited as a framework for this 
type of study because it makes the 
differences between novices and experts 
explicit (Tucker, 2014). Tucker’s 
subsequent work (Tucker, Weedman, Bruce, 
& Edwards, 2014) further develops the 
potential of a threshold concept approach to 
LIS education. 

Kiley and Wisker’s work on threshold 
concepts for doctoral researchers (2009) 
raises the question of whether information 
literacy may have threshold concepts that 
are bounded by a discipline, when the 
learning thresholds for research are present 
in every discipline. Brunetti, Townsend, and 
Hofer (2014) argue that the interdisciplinary 
nature of our teaching content indicates that 
information literacy threshold concepts need 
to be grasped by the student both in order to 
progress in her own field and to become 
information literate. Nevertheless, there are 
common ways of thinking and practicing 
shared by librarians—related to our own 
field, information science—that represent 
interdisciplinary learning thresholds that 
students can approach and cross (Townsend, 
Brunetti, & Hofer, 2011). 

Hofer, Townsend, and Brunetti (2012) 
conducted a qualitative survey in order to 
establish common “stuck places” for 
students, and to then extrapolate threshold 
concepts for information literacy—that is, 
concepts that students would need to grasp 
in order to get un-stuck. From this study, 
seven information literacy threshold 
concepts were proposed. Yet the study 
participants were so well-versed in the 
ACRL Information Literacy Competency 
Standards for Higher Education that the 
authors determined this shared mindset to be 
a significant limitation of the study. Using 
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the Delphi method, the present study 
addresses the limitations of the previous 
study and expands upon its findings. 

Methods 

The purpose of this study was to answer two 
questions: 

1. Is the threshold concept approach 
useful for information literacy? 

2. What are the threshold concepts 
for information literacy 
instruction? 

The Delphi Method 
The Delphi method was originally 
developed by the RAND corporation in the 
1950s to predict the future (it was named 
after the Greek oracle for this reason). A 
Delphi study is a qualitative research 
method in which a small group of experts 
are asked to anonymously answer questions 
about a topic in writing. It works in some 
ways like an extended group survey with 
opportunities to give feedback to others and 
revise individual answers. The Delphi 
method is not designed to generate proof for 
a theory backed by quantitative data. Rather, 
it brings a group of experts toward 
consensus around a given issue through an 
inherently qualitative process. Because 
threshold concepts are meant to be 
identified by experts in a given field, the 
Delphi method is a productive means by 
which to validate a group of proposed 
threshold concepts for infonnation literacy. 

Responses are collected and summarized by 
a moderator and then sent back to the 
experts. This process is called a round. In 
each round, experts read the responses of 
their peers, make adjustments to their own 
answers, and address questions raised 
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during the previous round. In this way, 
influence relating to professional reputation 
and personal demeanor is precluded. The 
purpose of conducting multiple rounds is to 
enable the experts to approach consensus on 
the research question (Luo & Wildemuth, 
2009). At the same time, as with other 
qualitative research methods, the process is 
just as informative as the end result: “Delphi 
may be seen more as a method for 
structuring group communication than 
providing definitive answers” (Charlton, 
2007, p. 246). 

Brian Cape’s article describing how he used 
a Delphi study in his Information and 
Library Management dissertation work 
(2004) was especially useful in informing 
the present study design (discussed further 
in the next section). In particular, Cape 
notes that “The way in which the results are 
fed back to the respondents can affect the 
final outcome... Producing the feedback was 
therefore the first stage in data analysis” (p. 
39). The authors found this to be the case; 
acting as moderators for material shared in 
each round was indeed a process of data 
analysis. Cape also emphasizes the 
importance of what he calls “member 
checking” of the outcomes: participants are 
invited to provide feedback on the results of 
the study, “ensuring] that the respondents 
agreed with the way in which the research 
had represented and interpreted their 
comments” (Cape, 2004, p. 45). As 
described below, the present study extended 
into an unexpected fourth round in order to 
check the outcomes with panelists. 

Forming a Panel of Experts 
Delphi study panelists are chosen based on 
their demonstrated expertise in the area of 
inquiry. For this study, panelists were 
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chosen based on their knowledge of and 
active participation in the field of 
information literacy and library instruction, 
as shown through publication, teaching, or 
leadership in professional organizations. 

An initial list of potential panelists was 
generated by a search of three databases: 
Library, Information Science & Technology 
Abstracts (LISTA), Library & Information 
Science Abstracts (LISA), and WorldCat. 
The search terms used were “information 
literacy” OR “library instruction” OR 
“research instruction” and the date range 
was limited to publications after 1995. This 
date range was selected in order to establish 
a list of experts who were likely to be 
currently active. The list of articles and 
books was exported, and authors with 
multiple publications or particularly well- 
cited, influential, or relevant publications 
were placed on a list. The leadership rosters 
of prominent organizations were also 
consulted, such as the ACRL Instruction 
Section and the IFLA Infonnation Literacy 
Section. From this list, a pool of 80 potential 
panelists were contacted with an email 
invitation to participate in the Delphi study. 
27 experts initially agreed to participate in 
the study through an online consent fonn, 
and 19 panelists participated in the first 
round. 

Research 

The present study was conducted from 
March 2013 to March 2014 and included: 

• formation of a panel of experts; 

• distribution of the Round 1 
questions; 

• analysis of Round 1 responses; 

• distribution of Round 1 analysis 
and Round 2 questions to panelists; 

• analysis of Round 2 responses; 
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• distribution of Round 2 analysis 
and Round 3 questions to panelists; 

• analysis of Round 3 responses & 
subsequent decision to continue for 
another round; 

• distribution of Round 3 analysis 
and Round 4 questions to panelists; 
and 

• analysis of Round 4 responses. 

Panelists were asked to begin by reading 
three articles. It was assumed that panelists 
possessed a thorough knowledge of 
information literacy, but may not have 
previously encountered threshold concepts. 
Therefore, panelists were asked to read two 
Meyer and Land articles about threshold 
concepts and one article concerning 
threshold concepts and infonnation literacy. 
Panelists were asked to consider two 
primary questions in each round; the 
research results therefore fall into two parts. 
Question 1 (Ql) considers the potential 
usefulness of a threshold concept approach 
to infonnation literacy. Questions 2 and 3 
(Q2 & Q3) identify and discuss infonnation 
literacy threshold concepts. 

Q1 of Round 1 began with a question about 
the viability of the threshold concepts 
approach for information literacy 
instruction, a simple yes/no question along 
with an invitation to discuss. Q2 of Round 1 
invited feedback on a list of potential 
threshold concepts, and Q3 asked panelists 
to suggest additional threshold concepts for 
information literacy. 

This pattern continued in all succeeding 
rounds. The Ql yes/no question on the 
usefulness of threshold concepts for 
information literacy instruction was 
answered decisively in Round 1 with a yes, 
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though the discussion of related issues 
continued in each round. Q2 for each round 
always began with a list of potential 
threshold concepts to discuss and Q3 asked 
panelists to suggest additional threshold 
concepts. After Round 1, panelists were 
asked to indicate which threshold concepts 
seemed strongest, and a ranked list was 
generated based on this feedback. The list of 
potential infonnation literacy threshold 
concepts and descriptions of those concepts 
was thus refined in each round based on 
participant feedback. 

Weaknesses 

The Delphi method has inherent 
weaknesses. The ability of the researchers to 
choose rather than sample for their experts 
affects the outcome of a Delphi study. Other 
limitations include the fallibility of experts, 
imprecision, and the bandwagon effect after 
the first round (Buckley, 1994). 

Though panelists were selected based on 
routine criteria for expertise (publishing, 
presenting, and participation in professional 
organizations), the composition of the panel 
inevitably reflects the demographics of 
academic librarianship in general. 
Additionally, panelists may have been more 
likely to agree to participate in the study if 
they knew one of the authors personally. 
The authors also had to make an extra effort 
to include practicing librarians, as 
publishing metrics alone could have resulted 
in a panel composed solely of LIS 
academics. 

Panelists were selected for their expertise in 
information literacy, as opposed to other 
areas of information science. Infonnation 
literacy experts typically do not have the 
technical skills possessed by librarians 
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working with digital collections, metadata, 
or other technical services. This lack of 
technical expertise may have influenced the 
type of threshold concepts that emerged 
from the study. 

This study was complicated by the release 
of the new ACRL Framework for 
Information Literacy between Rounds 3 and 
4 of the study. Several members of the task 
force were panelists in the Delphi study, and 
the Framework was initially based on early 
results from the Delphi study. Beyond this, 
the threshold concepts model as it relates to 
information literacy immediately became 
more prominent. The Framework may have 
served as an outside influence. 

The researchers initially planned for the 
study to run for three rounds, but at the end 
of Round 3, it became clear that some 
questions were unresolved and that another 
round would be needed. As a result, there 
was a significant delay of 3 months between 
Rounds 3 and 4, which may have affected 
the final results and amplified study fatigue 
among panelists. 

This study may also have been affected by 
the fact that the threshold concept model is 
itself a threshold concept, meaning it is 
difficult to understand and can take time to 
fully grasp. While the panelists were experts 
in information literacy, threshold concepts 
were new to some of them. Though 
unavoidable, this may have had an impact 
on the results of the study, as panelists spent 
time wrestling with their own understanding 
of the threshold concept model. 

Results 

As described in the Methodology section, 
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each round asked panelists to consider two 
questions: Ql, concerning the potential 
usefulness of the threshold concepts 
approach for information literacy, and Q2 
and Q3, identifying and evaluating proposed 
information literacy threshold concepts. 

Q1 was quickly answered in Round 1 with a 
unanimous “yes”: the threshold concepts 
approach holds potential for information 
literacy instruction. As described above, a 
detailed analysis of Q1 data will be made in 
a separate publication. 

Q2 & Q3 explored the viability of different 
proposed threshold concepts. In asking 
panelists to suggest and evaluate threshold 
concepts, the authors did not specify that the 
proposed concepts should meet all of the 
five definitional criteria. The six threshold 
concepts that emerged in the study vary in 
how fully they meet each criterion. 

The results presented here chart a course 
through a large collection of qualitative 
data. It is impossible to concisely relate the 
conversations that panelists engaged in 
about the various threshold concepts. 
However, it is these conversations that 
directed the development of each threshold 
concept. Data analysis was ongoing 
throughout the study in each successive 
round. The overall results can be presented 
as a linear description of this study’s Delphi 
process. The final list of proposed threshold 
concepts also functions as the results of the 
study. A link to the study data is posted at 
the website http://ilthresholdconcepts.com . 

Round 1 

Panelists were presented with seven 
proposed threshold concepts and asked to 
comment on them. Panelists were also asked 
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to propose threshold concepts of their own. 
Round 1 resulted in a list of 38 potential 
threshold concepts, including the original 
seven proposed by the authors, with 
descriptions ranging in length from a 
sentence to a substantial paragraph. 

The commentary about all of the threshold 
concepts was summarized and the list of 38 
potential threshold concepts with 
descriptions was returned to panelists for 
Round 2. 

Round 2 

Using the list of 38 potential threshold 
concepts generated in Round 1, panelists 
were asked to select the strongest and most 
compelling threshold concepts for 
information literacy. Panelists were again 
encouraged to suggest new potential 
threshold concepts. 

Round 2 generated discussion of the merits 
of various potential threshold concepts and 
15 additional proposed threshold concepts. 
Additionally, every time a participant 
included a threshold concept on his or her 
list of the most compelling proposed 
threshold concepts, it was tallied. Thus a 
ranked list of potential threshold concepts 
(Appendix A) was created based on this 
measure in order to track and organize 
panelist responses, though the rankings were 
not treated as formal quantitative data. 

Using the data generated in Round 2—the 
ranked list of potential threshold concepts, 
the descriptions of proposed threshold 
concepts, and participant discussion of the 
various proposed threshold concepts—the 
authors put together a list of nine potential 
information literacy threshold concepts. 
Each potential threshold concept included a 
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brief description and a list of proposed 
concepts that the authors attempted to 
combine into one definition. The table in 
Appendix B shows how the 38 threshold 
concepts proposed up to this point were 
reduced to nine. 

Round 3 

Panelists were asked to respond to the 
proposed list of nine infonnation literacy 
threshold concepts and descriptions. 
Panelists were again encouraged to propose 
new threshold concepts and suggested seven 
new concepts. 

All participant suggestions and comments 
about the proposed list of nine information 
literacy threshold concepts were placed on a 
spreadsheet for consideration. Another 
informal ranked tally of threshold concepts 
was generated (Appendix C). 

Based on the tally and comments, the 
authors combined proposed threshold 
concepts covering similar ground and 
distributed or strengthened other ideas 
throughout all of the threshold concepts. 
The list of proposed threshold concepts was 
subsequently shortened to six. 

Round 4 

The list of six proposed threshold concepts 
was returned to panelists for final 
comments. It was understood that this 
would be the final round. For the purposes 
of this study, this was “member checking,” 
as described by Cape (2004). 

The majority of comments, however, 
concluded that the list of threshold concepts 
generated in Round 3 was useful and 
represented some important understandings 
in infonnation literacy. Remaining feedback 
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was incorporated into the final descriptions 
of six threshold concepts. 

Proposed Information 
Literacy Threshold Concepts 

The final results of the Delphi study are a 
list of proposed threshold concepts for 
information literacy. The following terms 
are used throughout the threshold concept 
descriptions. 

The authors view information literacy as 
competence in working with systems of 
information to discover, evaluate, manage, 
and use infonnation effectively in context, 
informed by an understanding of the social, 
political, cultural and economic dimensions 
that affect the creation and dissemination of 
information within those systems. Much like 
other new literacies (e.g., financial, 
mathematical, visual), infonnation literacy 
can be understood as a facility with the 
foundational concepts of a given area of 
inquiry, in this case infonnation science, 
and the ability to apply those understandings 
and skills in other areas of life. 

Expert refers to librarians, information 
scientists, or others with substantial 
knowledge in the field of information 
science. Though threshold concepts 
represent expert understandings and 
practice, moving through a given set of 
threshold concepts does not necessarily 
make one an expert. Beginners or novices 
are people new to the field. Faculty or 
researchers in other disciplines are not 
assumed to be experts in the field of 
information science. Likewise, not all 
librarians will be conversant with all the 
details of the various threshold concepts, 
depending on their specialization within 
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librarianship. 

Finally, one of the characteristics of 
threshold concepts is that they are 
integrative, and therefore, the content of the 
threshold concepts proposed here sometimes 
overlaps. 

Authority 

Authoritative evidence comes from sources 
that possess the expertise, experience, and 
relevant credentials to be considered 
trustworthy. However, those criteria are not 
constant across settings or situations; the 
disciplines have differing views of what 
constitutes evidence, and different situations 
give rise to different criteria for evaluation 
of authority, whether acknowledged or 
implicit. People create authoritative 
evidence as well; an information need might 
not be met by existing evidence. Examining 
the characteristics of authoritative evidence 
in specific contexts illuminates the systems 
that grant authority, including their faults, 
along with considerations of when, where, 
and why these systems are used. Understood 
in this way, authority is a reflection of 
societal structures of power. 

• Transformative: The learner’s 
understanding is transformed to a 
more complex understanding of 
authoritative evidence in which its 
utility shifts depending on how it 
is being used and the questions 
being answered. 

• Irreversible: Experts hold a 
nuanced view of authority that is 
not conferred by simple or static 
markers. 

• Integrative: This threshold 

concept helps a learner understand 
the fonnat a creator may choose, 
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the commercial and ethical 
implications of credibility, and the 
ways by which scholarly 
conversation can elevate or 
demote a piece of evidence. 

• Troublesome: Novices may 

understand evidence and authority 
as unchangeable and can struggle 
to relate their own use of evidence 
in daily life to scholarly or 
professional approaches to 
evidence. 

• Bounded: This threshold concept 
is not bounded by information 
science. However, librarians are 
concerned with how authority 
facilitates or limits the movement 
of information through systems of 
production and dissemination. 

Format 

Information is packaged in different formats 
because of how it was created and shared. 
Focusing on process de-emphasizes the 
increasingly irrelevant dichotomy between 
print and online sources by examining 
content creation in addition to how that 
content is delivered or experienced. While 
the relevance of the physical characteristics 
of various fonnats has waned with the 
increasing availability of digital 
information, understanding fonnat in the 
context of the infonnation cycle is still an 
essential part of evaluating infonnation. 
Critical questions can be asked about 
content and how and why it was produced. 
Understanding who has access to publishing 
via different fonnats, and which voices are 
heard or silenced in different 
communication channels, reveals a great 
deal about power structures and privilege. 

• Transformative: The learner’s 
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understanding is transformed 
because understanding the pattern 
of events which produce 
infonnation fundamentally 

changes the novice’s view of 
infonnation as a flat, 
undifferentiated landscape served 
up in a browser window. Instead, 
learners select information by 
looking to the processes and 
structures governing infonnation 
production. 

• Irreversible: Experts do not see 

different formats as 

interchangeable or identical. 

• Integrative: This threshold 

concept brings together lessons 
about source selection, 
infonnation evaluation, and 
citation. 

• Troublesome: Novices may have 
preconceived ideas about the 
value of certain formats. It also 
may represent a language problem 
for beginners who are accustomed 
to using the word “website” to 
mean “I found it online” and are 
now asked to use a specific and 
narrower meaning. Finally, 
because the current infonnation 
landscape has stripped sources of 
the clues that physical format used 
to offer, sources are increasingly 
difficult to categorize. 

• Bounded: Format has long 
represented the final stage of 
infonnation dissemination and has 
dictated much of the structure of 
systems for retrieval and storage. 

Information Commodities 

The cost of information, academic or 

otherwise, is often obscured. Infonnation 
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may appear to be free because libraries 
negotiate subscriptions or interlibrary loans, 
institutional repositories and open-access 
journals do not charge for their services, and 
a deluge of infonnation is brought up by a 
web search. Yet costs are associated with 
information production, and revenue may be 
generated as a result of its use. 
Understanding these realities can encourage 
critical thinking and resistance around the 
implications of the commodification of 
information—for example, privacy, filter 
bubbles, net neutrality for web content, and 
personal data. Considering the financial 
relationships involved in information 
production, consumption, and dissemination 
allows for thoughtful choices about 
information sources and personal data while 
prompting questions about the economic 
and proprietary influences that impact 
information flow. 

• Transformative: The learner’s 
understanding is transfonned 
when the reasons behind barriers 
to information (such as multiple 
logins, embargos on current 
issues, or pop-up advertisements) 
are examined. It explains the 
purpose guiding academic 
practices such as attribution, 
authentication for databases, or 
publication expectations for 
faculty. The act of using 
infonnation is exposed as an 
economic and political choice that 
requires care and consideration. 

• Irreversible: Experts understand 
the value of infonnation and do 
not consider any information to be 
unequivocally free; they also may 
understand the issue from the 
perspective of a content creator as 
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the author of published work. 

• Integrative: This threshold 

concept links the academic 
experience to other familiar 
situations that involve buying and 
selling goods, while extending the 
research process beyond the 
classroom; this concept connects 
the novice researcher to a wider 
network of infonnation producers: 
scholars, agencies, institutions, 
and corporations. 

• Troublesome: Much of the 

infonnation available to novices 
comes without a direct cost. Still, 
infonnation is sold, bought, and 
requires labor to produce. Given 
the philosophical motives for 
open-access publishing, 

institutional repositories, open 
educational resources, and efforts 
to reduce the digital divide, this 
threshold concept may also 
introduce questions about the 
point at which infonnation is not 
only a commodity, but also a 
human right. 

• Bounded: Librarians have a 
unique perspective on the 
commodification of infonnation 
because of our role as advocates 
for broad access to infonnation 
and purchasers on behalf of our 
communities. 

Information structures 
Opening the hood on databases and search 
engines transfonns them from mysterious 
boxes that magically produce good-enough 
information on command into systems that 
can be used precisely and efficiently. 
Infonnation users leverage database features 
such as field searching, controlled 
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vocabulary, and filtering to retrieve 
appropriate materials. Infonnation creators 
organize infonnation for inclusion in 
information systems, and also design such 
systems, whether managing personal 
information or disseminating research data 
for re-use. Though information structures 
are highly dependent upon technology, the 
underlying principles of organization and 
classification are still largely about 
organizing knowledge, mediated by fonnat. 
Because people structure information and 
the systems that contain it—and human 
knowledge is contested, negotiated, and 
continually evolving—information 

structures often reflect economic, 
disciplinary, and social conventions rather 
than adhering to strictly logical principles of 
organization. 

• Transformative: The learner’s 
understanding is transformed 
when the structures that make 
infonnation findable are explored. 
Questioning what content is in the 
database being searched 
complicates the idea that a single 
search tool can serve up all the 
infonnation to meet a need. 

• Irreversible: Experts structure data 
so that it can be reused 
effectively. They do not treat the 
search box as simple or magical. 

• Integrative: This threshold 

concept integrates common 
lessons such as brainstorming 
keywords, Boolean operator 
activities, exploration of subject 
databases, and the value 
proposition of the library as 
opposed to web search. 

• Troublesome: Novices must leave 
the comfort zone of their prefened 
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search strategy. Searching within 
complex infonnation structures 
requires effort, patience, and 
persistence. 

• Bounded: Librarians are trained 
in, care about, and often create 
database structures and search 
interfaces for infonnation 
retrieval. 

Research Process 

Identifying and articulating useful research 
questions requires preexisting knowledge 
and is difficult intellectual work. Applying 
information to a problem, or using it as 
evidence in an argument or for inspiration in 
a creative endeavor, requires that the 
researcher understand what will qualify as 
disciplinary evidence. This process of 
inquiry, research, and use is one of iterative 
inquiry, allowing for mistakes and 
correction of earlier misapprehensions. 
From inquiry to seeking out existing 
knowledge, to the selection of relevant 
information, to the development and testing 
of a thesis/hypothesis and subsequent 
analysis and synthesis of the results, the 
process results in the creation of new 
knowledge. Engaging in the infonnation 
creation process is an extension of the 
thinking process, and therefore “research” 
may be understood as a broadly 
encompassing tenn—though some fonns of 
research may be more or less valued in 
academia. 

• Transformative: The learner’s 
understanding is transformed 
when research is positioned as one 
means by which new knowledge 
may be created. Research is no 
longer simply the retrieval and 
compilation of discrete facts about 
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a topic but is used to solve 
problems and answer questions 
both within and beyond the 
library. 

• Irreversible: Experts are willing to 
work through new understandings 
as infonnation is gathered and 
analyzed. They use these 
understandings to develop and 
refine a topic of inquiry because 
one of the primary purposes of 
research is to reach new 
understandings, not necessarily to 
confirm old ones. 

• Integrative: The research process 
brings together the skills 
necessary for developing a thesis 
or topic and combines them with 
those required for finding and 
communicating information. 

• Troublesome: Novices may think 
that asking questions should be 
easy; good questions may be 
perceived as springing forth 
whole from the creative mind. It 
may seem like a waste of time to 
do background research solely in 
order to get to the point of being 
able to ask a question. 

• Bounded: This threshold concept 
is not bounded by information 
science. However, librarians are 
familiar with varied paths of 
inquiry that span across 
disciplines and are well positioned 
to offer insight on how to 
structure a question, where to ask 
it, and how to adjust a question 
based on new information. 

Scholarly Discourse 

Infonnation users and creators are part of an 

ongoing conversation in which new 
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knowledge builds upon or refutes what has 
gone before, and in turn inspires others. 
Knowledge is negotiated through ongoing 
discourse. In some cases, close study of 
existing conversations will lead to a new 
inquiry as a literature review reveals gaps in 
the conversation. In fact, scholarly discourse 
is most compelling when it is approached 
with a research question in mind. As an 
extension of scholarship as a conversation, 
scholarly conversation and knowledge 
creation take place in the context of a 
community that includes novices, 
apprentices, and experts. Communities 
uphold standards and exert influence on the 
content produced within those guidelines; 
communities may also resist new or 
dissenting understandings. Some 
communities may be difficult for certain 
populations to access, depending on the 
expectations of the community, the cost of 
entry, or social barriers. 

• Transformative: The learner’s 
understanding is transformed 
when the novice in the classroom 
is connected to thinkers and 
creators that transcend space and 
time. 

• Irreversible: Experts do not treat 
their work as though it were 
produced in a vacuum. Read in 
this way, the bibliography of a 
scholarly paper becomes a point 
of access and citation has a 
function beyond the negative 
purpose of avoiding plagiarism. 

• Integrative: This threshold 

concept reveals scholarly 

conventions that novices may 
have learned or observed to be 
part of an academic culture with 
specific (though often unspoken) 
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rules. 

• Troublesome: Novices may find it 
uncomfortable to consider 
knowledge as negotiated rather 
than fixed; they may struggle to 
connect their work to the broader 
conversations in the discipline. 
They have to let go of the trope of 
the lone genius with a light bulb 
over his or her head. 

• Bounded: This threshold concept 
is not bounded by information 
science. Yet, libraries and 
associated systems of information 
storage and retrieval have 
historically been charged with 
providing access to the records of 
scholarship and cultural heritage. 

Discussion 

Infonnation literacy is an application of 
information science, and information 
science is an interdisciplinary field. As such, 
the boundaries of our discipline may be 
difficult to locate or may overlap with those 
of other disciplines. We may not be the only 
discipline concerned with scholarly 
discourse or contextual authority, but our 
approach to these topics is distinct. The 
threshold concept model also leaves room 
for divergent thinking about information 
literacy topics within the field. 

Traditional bibliographic instruction 
positions the librarian as a supplementary 
source of expertise to the subject faculty and 
as a gatekeeper for scholarly resources. 
Identifying information literacy threshold 
concepts repositions the librarian as a 
subject matter expert and explicitly defines 
the content areas that are bounded by 
information literacy. As subject matter 
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experts with big ideas to teach, it follows 
that librarians need more than a 50-minute 
one-shot session with students. A credit 
course provides enough time to at least 
introduce students to learning thresholds, 
even if they might not make it all the way 
across the threshold in a term. 

At the same time, librarians still have 
important procedural information to convey 
that can help students master the rules of the 
academic game. This infonnation— 
bibliographic instruction—has been shown 
in many studies to help students succeed in 
higher education (for example, Cook, 2014). 
When it is coupled with underlying big 
ideas or threshold concepts, students have 
the chance to integrate the discrete points 
and gain a deeper understanding. 

Because threshold concepts uncover the 
tacit knowledge of a discipline they can 
open an explicit examination of the 
assumptions behind the disciplinary lens 
that we ask students to look through. This 
enables instructors to acknowledge and 
situate their own perspectives, biases, 
values, and ideologies and invites students 
to evaluate the point of view for themselves 
in deciding whether to adopt it. In the case 
of librarians, it can help us articulate what it 
might mean to make “little librarians” out of 
our students; that is, to help them become 
information literate. 

Encouraging students to use our disciplinary 
lens need not be an act of conformity or 
assimilation. There is room within threshold 
concepts to re-examine nonnative 
assumptions about the academic or 
information world. Because the authors 
have primarily worked for institutions with 
significant populations of underserved 
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students—first-generation, nontraditional, 
community college, or underrepresented 
minorities—our stance is that threshold 
concepts should be used to encourage 
students to engage critically with the content 
that we teach in order to assist them in 
defying structural expectations. This view is 
consistent with the personal experiences that 
the authors bring to our professional lives, 
which are informed by the various ways in 
which we do not fit the presumed 
demographic for academics (straight, white, 
and male). 

The threshold concepts generated by this 
study are not meant as a comprehensive 
outline of instructional content for 
information literacy. The authors expect that 
more threshold concepts will emerge as 
more practitioners engage with the threshold 
concept model. Threshold concepts will 
exist for specific areas of information 
science, such as metadata and discovery, 
and be articulated by librarians not 
traditionally associated with library 
instruction. While there is certainly room to 
expand on the present findings, the authors 
do believe that the threshold concepts 
identified by the Delphi panelists accurately 
describe six of the big ideas underlying the 
content that we teach. These threshold 
concepts can help a novice view information 
through a librarian lens. 

Conclusion 

As Box and Draper point out in their 
seminal work on model-building, “...all 
models are wrong; the practical question is 
how wrong do they have to be to not be 
useful” (p. 74). The threshold concepts 
model is imperfect and practitioner 
understanding of that model is likewise 
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flawed. However, this study collected the 
consensus of a group of expert practitioners 
and they found the model to be useful. 
Likewise, the authors have found the model 
to be useful in the real world when teaching 
information literacy. 

Drawing upon the words of infonnation 
literacy advocate Bill Badke: 

Educators are going to need to move 
from teaching about their disciplines 
to enabling their students to become 
disciplinarians... We must invite 
students into our world and there 
reproduce ourselves in them, turning 
our students into active practitioners 
in our disciplines. (2012, p. 93) 

The purpose of this study was not to 
discourage librarians from teaching research 
skills by taking learners through library 
databases or subject headings. Instead, the 
study asked librarians to situate crucial 
skills within larger conceptual 
understandings, and to consider how we, as 
instructors, can reach back into the long- 
gone versions of our novice minds in order 
to show learners how to operate as 
practitioners in our discipline. 

Authors’ Note 

The authors would like to thank the Delphi 
study participants for their substantial 
commitment of time and effort to this study. 
We are also grateful to Joe Delwiche, Jen 
Hofer, David Hurley, Lora Leligdon, Conor 
Lin Hanick, James Markham, and Robert 
Schroeder for their help in developing ideas 
for this paper. 


[Article] 


39 





Townsend, et al, Identifying Threshold Concepts 

References 

Association of College & Research 
Libraries. (2014). Framework for 
information literacy for higher education. 
Retrieved from http://www.ala.org/acrl/ 
standards/ilframework 

Atherton, J., Hadfield, P., & Meyers, R. 
(2008). Threshold concepts in the wild. 
Expanded paper presented at the Threshold 
Concepts: From Theory to Practice 
conference, Kingston, Ontario. Available 
from http://www.doceo.co.uk/tools/ 

Threshold Concepts Wild expanded 70.pdf 

Badke, W.B. (2012). Teaching research 
processes: The faculty role in the 
development of skilled student researchers. 
Oxford: Chandos Publishing. 

Barradell, S. (2013). The identification of 
threshold concepts: A review of theoretical 
complexities and methodological 
challenges. The International Journal of 
Higher Education Research, 65(2), 265- 
276. doi: 10.1007/s 10734-012-9542-3 

Baruchson-Arbib, S., & Bronstein, J. 

(2002). A view to the future of the library 
and information science profession: A 
Delphi study. Journal of the American 
Society for Information Science and 
Technology, 53(5), 397-408. doi: 10.1002/ 
asi. 10051 

Bent, M., Gannon-Leary, P., & Webb, J. 
(2007). Information literacy in a researcher's 
learning life: The seven ages of research. 
New Review of Information Networking, 13 
(2), 81-99. 

doi: 10.1080/13614570801899983 
Blackmore, M. (2010). Student engagement 


Communications in Information Literacy 10(1), 2016 

with information: Applying a threshold 
concept approach to information literacy 
development. Paper presented at the Third 
Biennial Threshold Concepts Symposium, 
Sydney, Australia. Available from http:// 
unsworks.unsw. edu.au/ fapi/datastream/ 

unsworks: 8 914/S OURCEO1 

Blackmore, M. & Freeland, P. (2014). 
Serious play: Threshold concepts, 

information engagement and game design. 
In C. O’Mahony, A. Buchanan, M. 
O’Rourke, & B. Higgs (Eds.), Threshold 
concepts: From personal practice to 
communities of practice, Proceedings of the 
National Academy's Sixth Annual 
Conference and the Fourth Biennial 
Threshold Concepts Conference. Available 
from http://www.nairtl.ie/workgroupDocs/ 
BlackmoreFreeland.pdf 

Box, G. E. P., & Draper, N. R. (1987). 
Empirical model-building and response 
surfaces. New York: Wiley. 

Buckley, C.C. (1994). Delphi technique 
supplies the classic result? The Australian 
Library Journal, 43(3), 158-164. 

doi: 10.1080/00049670.1994.10755684 

Buehler, M. A., & Zald, A. E. (2013). At the 
nexus of scholarly communication and 
information literacy: Promoting graduate 
student publishing success. In S. Davis-Kahl 
& M.K. Hensley (Eds.), Common ground at 
the nexus of information literacy and 
scholarly communication (pp. 215-235). 
Chicago: Association of College & 
Research Libraries. 

Cape, B. (2004). Gathering opinion and 
initiating debate: The success of the Delphi 
method in purely qualitative research. 


[Article] 


40 


















Townsend, et al, Identifying Threshold Concepts 

Library and Information Research, 25(89), 
35-44' 

Charlton, J. (2004). Delphi technique. In M. 
Lewis-Beck, A. Bryman, & T. Liao (Eds.), 
Encyclopedia of social science research 
methods (pp. 245-246). Thousand Oaks, 
CA: SAGE Publications, Inc. 

doi: 10.4135/9781412950589.ii219 

Cook, J. M. (2014). A library credit course 
and student success rates: A longitudinal 
study. College & Research Libraries, 75(3), 
272-283. doi: 10.5860/crll2-424 

Cousin, G. (2008). Threshold concepts: Old 
wine in new bottles or a new form of 
transactional curriculum inquiry? In R. 
Land, J.H.F. Meyer, & J. Smith (Eds.), 
Threshold concepts within the disciplines 
(pp. 261-272). Rotterdam: Sense 

Publishers. 

Dixon-Thomas, C. (2012). Information 
literacy and the 21st century academic 
librarian: A Delphi study (Doctoral 
dissertation). Available from ProQuest 
Dissertations and Theses database. 

Doyle, C. S. (1992). Summary of findings: 
Outcome measures for information literacy 
within the national education goals of 1990. 
Available from http://files.eric.ed.gov/ 
fulltext/ED3 51033.pdf 

Feret, B., & Marcinek, M. (2005). The 
future of the academic library and the 
academic librarian: A Delphi study 
reloaded. New Review of Information 
Networking, 77(1), 37-63. 
doi: 10.1080/13614570500268381 

Flanagan, M. (2014). Threshold concepts: 


Communications in Information Literacy 10(1), 2016 

Undergraduate teaching, postgraduate 
training and professional development. A 
short introduction and bibliography. 
Retrieved from http://www.ee.ucl.ac.uk/ 
- m 11 an a ga/th re sh o lds.html 

Fuzzard, R., & Kiley, M. (2013) What 
threshold concepts underpin skills training 
in community services in vocational 
education and training? Paper presented at 
the 4th Threshold Concepts Conference, 
Dublin. Available from http://www.nairtl.ie/ 
workgroupDocs/FuzzardWiley.pdf 

Green, J. W. (2000). Delphi method in web 
site selection: Using the experts. Reference 
Librarian, 33(69/70), 299-310. 

doi: 10.1300/J120v33n69 27 

Hofer, A.R., Brunetti, K., & Townsend, L. 
(2013). A threshold concepts approach to 
the Standards revision. Communications in 
Information Literacy, 7(2), 108-113. 

Available from http:// 

www.comminfolit.org/index.php? 
iournal=cil&page=article&op=view&path% 

5B%5D=v7i2p 108&path%5B%5D=l 68 

Hofer, A.R., Townsend, L., & Brunetti, K. 
(2012). Troublesome concepts and 
information literacy: Investigating threshold 
concepts for IL instruction, portal: Libraries 
and the Academy, 72(4), 387-405. Available 
from http://archives.pdx.edu/ds/psu/8542 

Howze, P.C., & Dalrymple, C. (2004). 
Consensus without all the meetings: Using 
the Delphi method to determine course 
content for library instruction. Reference 
Services Review, 32(2), 174-184. 

doi: 10.1108/00907320410537694 

Kiley, M., & Wisker, G. (2009). Threshold 


[Article] 


41 




















Townsend, et al, Identifying Threshold Concepts 

concepts in research education and evidence 
of threshold crossing. Higher Education 
Research & Development, 28(4), 431-441. 
doi: 10.1080/07294360903067930 

Land, R., Meyer, J., & Smith, J. (2008). 
Threshold concepts within the disciplines. 
Rotterdam: Sense Publishers. 

Loughran, J., Berry, A., & Mulhall, P. 
(2006). Understanding and Developing 
Science Teachers' Pedagogical Content 
Knowledge. Rotterdam: Sense Publishers. 

Lundstrom, K., Fagerheim, B. A., & 
Benson, E. (2014). Librarians and 
instructors developing student learning 
outcomes: Using frameworks to lead the 
process. Reference Services Review, 42(3). 
doi: 10.1108/RSR-04-2014-0007 

Luo, L., & Wildemuth, B. M. (2009). 
Delphi studies. In Wildemuth, B. M. (Ed.), 
Applications of social research methods to 
questions in information and library science 
(pp. 83-92). Westport, CT: Libraries 
Unlimited. 

Meyer, J., & Land, R. (2003). Threshold 
concepts and troublesome knowledge: 
Linkages to ways of thinking and practising 
within the disciplines. (ETL Project 
Occasional Report 4). Edingurgh: 
Enhancing Teaching-Learning 

Environments in Undergraduate Courses 
Project. Available from http:// 
www.colorado.edu/ftep/documents/ 

ETLreport4-1 .pdf 

Meyer, J., & Land, R. (2006). Overcoming 
barriers to student understanding: 
Threshold concepts and troublesome 
knowledge. London: Routledge. 


Communications in Information Literacy 10(1), 2016 

Meyer, J., & Land, R. (2007). Stop the 
conveyor belt, I want to get off. Times 
Higher Education Supplement, 1807. 
Available from http:// 

www.timeshighereducation. co .uk/9028 8 .arti 

cle 

Meyer, J., Land, R., & Baillie, C. (2010). 
Threshold concepts and transformational 
learning. Rotterdam: Sense Publishers. 

Middendorf, J., & Pace, D. (2004). 

Decoding the disciplines: A model for 
helping students leam disciplinary ways of 
thinking. New directions for teaching and 
learning, 2004(9%), 1-12. doi: 10.1002/ 

tl.142 

Neuman, D. (1999). High school students’ 
use of databases: Results of a national 
Delphi study. Journal of the American 
Society for Information Science, 46(4), 284- 
298. doi: 10.1002/(SICI) 1097-4571(199505) 
46:4<284::AID-ASI5>3.0.CO;2-J 

Nguyen, Q. H. (2012). Everyday threshold 
concepts: Implications for sustainable 
agriculture education in Vietnam’s Mekong 
Delta. In INTED2012 Proceedings, 4089- 
4098. Available from http://www.zef.de/ 
uploads/tx zefportal/ 

Publications/2e29 Nguyen% 

20INTED2012%20paper.PDF 

Nicola-Richmond, K. (2014). The 
transformation from student to occupational 
therapist: Using the Delphi method to 
identify the threshold concepts in 
occupational therapy. Paper presented at 
the Fifth Biennial Threshold Concepts 
Conference, Durham, UK. Available from 
http://www.ee.ucl.ac.uk/~mflanaga/ 

abstracts/TC 14Abstract6.pdf 


[Article] 


42 





















Townsend, et al, Identifying Threshold Concepts 

Oakleaf, M. (2014). A roadmap for 
assessing student learning using the new 
Framework for Information Literacy for 
Higher Education. Journal of Academic 
Librarians hip, 40(5), 510-514. doi: 10.1016/ 
i.acalib.2014.08.001 

Rodrigues, D. B., & Sedo, D. N. (2008). 
Experiencing information literacy in Second 
Life. Partnership: The Canadian Journal of 
Library and Information Practice and 
Research, 5(1). Available from http:// 
ioumal.lib.uoguelph.ca/index.php/perj/ 

article/view/426/860 

Rowbottom, D. P. (2007). Demystifying 
threshold concepts. Journal of Philosophy of 
Higher Education, 41(2), 263-270. 

doi: 10.1 111/i. 1467-9752.2007.00554.X 

Saracevic, T. (2014). Information literacy in 
the United States: Contemporary 

transformations and controversies. Paper 
presented at the European Conference on 
Information Literacy, Dubrovnik, Croatia. 
Available from http://comminfo.mtgers.edu/ 
~tefko/Saracevic%20ECIL%202014% 

20Inf%201iteracy%20in%20US% 

20Springer.pdf 

Saunders, L. (2009). The future of 
information literacy in academic libraries: A 
Delphi study, portal: Libraries and the 
Academy, 9(1), 99-114. 

Seeker, J. (2011). A new curriculum for 
information literacy: Expert consultation 
report. Available from http:// 
ccfil.pbworks.com/frExpert report final.pdf 

Seeker, J., & Coonan, E. (2012). 

Developing a new curriculum for 
information literacy: Reflections on our 


Communications in Information Literacy 10(1), 2016 

Arcadia fellowship research. ALISS 
Quarterly, 7(2), 20-22. 

Shinners-Kennedy, D., & Fincher, S. A. 
(2013, August). Identifying threshold 
concepts: From dead end to a new direction. 
In Proceedings of the ninth annual 
international ACM conference on 
International computing education 
research, doi: 10.1145/2493394.2493396 

Townsend, L., Bmnetti, K., & Hofer, A.R. 
(2011). Threshold Concepts and 
Information Literacy, portal: Libraries and 
the Academy, 11(3). Available from http:// 
archives.pdx.edu/ds/psu/7417 

Tucker, V. (2012). Acquiring search 
expertise: Learning experiences and 

threshold concepts (Doctoral dissertation). 
Queensland University of Technology, 
Brisbane, Australia. Available from http:// 
www.virginiatucker.com/docs/ 

virginiatucker thesis finala4.pdf 

Tucker, V. M. (2014). The expert searcher’s 
experience of information. In C. Bmce, K. 
Davis, H. Hughes, H. Partridge, & I. 
Stoodley (Eds.), Information experience: 
Approaches to theory and practice (pp. 226 
-241). Bingley, UK: Emerald Group 
Publishing Limited. 

Tucker, V. M., Weedman, J., Bmce, C. S., 
& Edwards, S. L. (2014). Learning portals: 
Analyzing threshold concept theory for LIS 
education. Journal of Education for Library 
and Information Science, 55(2), 150-165. 

Wen, J. R., & Shih, W. L. (2006). Exploring 
the information literacy competence 
standards for elementary and high school 
teachers. Computers & Education, 50 


[Article] 


43 






















Townsend, et al, Identifying Threshold Concepts 


Communications in Information Literacy 10(1), 2016 


(2008), 787-806. doi: 10.1016/ 

j.compedu.2006.08.011 

Wiggins, G., & McTighe, J. (2005). 
Understanding by design. Alexandria, VA: 
ACD. 

Wilkinson, L. (2014). The problem with 
threshold concepts [Web log post]. Sense 
and Reference. Available from https:// 
senseandreference.wordpress.com/2014/06/ 

19/the-problem-with-threshold-concepts/ 

Xiaomu, Z., Ping, S., Mengli, W., & 
Weichun, D. (2008). Delphi research on 
information literacy competency standards 
for higher education in Beijing, China. 
Chinese Librarians hip: An International 
Electronic Journal, 25. Available from 
http://white-clouds.com/iclc/cliej/ 

cl25ZSWD.pdf 

Yorke-Barber, P., Atkinson, L., Possin, G., 
& Woodall, L. (2008). Light bulb moments: 
Identifying information research threshold 
concepts for fourth year engineering 
students. Proceedings of the 2008 AaeE 
Conference. Available from http:// 
espace.library.uq.edu.au/eserv/UQ: 159735/ 

aaee08 submission TlC2.pdf 


44 


[Article] 














Townsend, et al, Identifying Threshold Concepts 


Communications in Information Literacy 10(1), 2016 


Appendix A—Round l Tally of Proposed Threshold Concepts 


Proposed threshold concept 


Authority is constructed and contextual 


Scholarship is a conversation 


Searching is not magic 


Fonnat as a process 


Information as a commodity 


Infonnation is socially constructed and is created and functions 
within existing power structures 


The Nature of evidence is disciplinary 


Research involves a community 


Research answers questions 


Research is conversation 


Research solves problems 


Research is a process 


Differentiating between data, information, knowledge, and 
wisdom 


“Primary source” is an exact and conditional category 


Everything has bias 


Student as producer of information 


Personal belief underpins information processing 


Absence of evidence is not evidence of absence 


Information is not something to be gathered and learned from, 
but rather something that you engage with and that personally 
transforms in a creative learning journey 


Deep commitment to access to information and intellectual 
freedom and the ability of all to have experiences of knowledge 


Integration of sources in synthesis and creation of new 
infonnation 


Collections are organized by conventions 


Information is created by people 


Endorsed as threshold 
concept by participant 
(out of 14 possible) 3 


13 
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Appendix A—continued 


Proposed threshold concept 

Endorsed as threshold 
concept by participant 
(out of 14 possible) 3 

All categories are conditional and constructed 

3 

You won’t find everything on Google 

3 

Information can be discipline-dependent 

2 

Information apprenticeship in community 

2 

Information as a political force 

2 

Personal profile and identity 

2 

There are systems at work here and you can leam to use them 

2 

You can’t search everything the same way you search Google 

2 

The user and the creator 

2 

There are always more than two sides 

1 

Honesty in the information landscape 

1 

It’s not what you say it’s the way that you say it 

1 

Constructedness of the systems and communities and their 
embeddedness in political, economic, and social contexts 

1 


a 17 people responded, 3 people didn't choose specific threshold concepts, so that leaves 14 as total number 
possible in Column B 
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Appendix B—Round 2 List of Proposed Threschold Concepts 


Proposed Threshold 
Concept (working title) 

Concepts included 

Evidence changes 
depending on context (e.g. 
disciplinary) 

Information is constructed for specific purposes 

Every resource has its use 

“Primary source” is an exact and conditional category 

The Nature of evidence is disciplinary 

Information can be discipline-dependent 

Texts will have different meanings in different social/political/ 
scientific and/or historical contexts 

Authority is constructed and 
contextual 

Authority is constructed and contextual 

Research is a process of 
inquiry and creates new 
knowledge 

Academic libraries are in knowledge creation business 

Research solves problems 

Research answers questions 

Research is a process 

Research facilitates inquiry 

Searching is not magic 

Searching and finding is not a linear process 
Metadata=Findability 

Good searches use database structure 

You won't find everything 
in one place 

Expert pays attention to gaps and uses multiple resources and 
strategies to fill gaps 

You won’t find everything on Google 

You can’t search everything the same way you search Google 
First results and initial findings are exactly that-first and initial 

Format is a process 

Fonnat is a process 

Infonnation as a commodity 

Once created, infonnation is usually owned and must be used 
within the constraints inherent in that ownership 

Scholarship is a 
conversation 

Research is conversation 

Scholarship is a conversation 
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Appendix B—continued 
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Concepts included 

Information is not something to be gathered and learned from, 

but rather something that you engage with and that personally 

transforms in a creative learning journey 

Personal profile and identity 

Personal belief underpins information processing 

Everything has bias 

There are always more than two sides 

Information is created by people 

Collections are organized by conventions 

Honesty in the infonnation landscape 

Information apprenticeship in community 

Research involves a community 

Information is socially constructed and is created and functions 
within existing power structures 

Constructedness of the systems and communities and their 
embeddedness in political, economic, and social contexts 
Information as a political force 
All categories are conditional and constructed 


Proposed Threshold 
Concept (working title) 

Infonnation is socially 
constructed 
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Appendix C—Round 3 Tally ol Proposed Threshold Concepts 

Yes 
11 
10 

7 
9 
7 
9 

11 
9 
5 
1 
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Is this a threshold concept? 

Authority is constructed and contextual 

Evidence changes depending on context (e.g. 
disciplinary) 

Format is a process 

Infonnation as a commodity 

Infonnation is socially constructed 

Research is a process of inquiry and creates new 
knowledge 

Scholarship is a conversation 
Searching is not magic 
You won't find everything in one place 
Infonnation apprenticeship 
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